Papers with multimodal emotion recognition
Dynamic Graph Neural ODE Network for Multi-modal Emotion Recognition in Conversation (2025.coling-main)
Copied to clipboard
| Challenge: | Existing graph-based multimodal emotion recognition methods fail to capture dynamic changes in emotions. |
| Approach: | They propose a Dynamic Graph Neural Ordinary Differential Equation Network (DGODE) which combines dynamic changes of emotions to capture temporal dependencies of speakers’ emotions. |
| Outcome: | The proposed model can capture the temporal dependencies caused by dynamic changes in emotions and can improve on two publicly available multimodal emotion recognition datasets. |
CMTD: Cognitive Modeling with Traits and Distortions for Multimodal Emotion Recognition in Conversations (2026.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that traits temper negativity bias from distortions, and cognitive modeling with psychological, visual, and acoustic information can improve the performance of MERC. |
| Approach: | They propose a framework for multimodal emotion recognition in conversations that takes advantage of stable personality traits, dynamic cognitive distortions, visual and acoustic features of interlocutors to enhance the emotional intelligence of LLMs. |
| Outcome: | Experimental results show that traits temper negativity bias from distortions, and cognitive modeling with psychological, visual, and acoustic information can improve the performance of MERC. |
Self-adaptive Context and Modal-interaction Modeling For Multimodal Emotion Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to predict emotion label for a given utterance lack modeling of diverse dependency ranges and inconsistent treatment of contribution for various modalities. |
| Approach: | They propose a multimodal emotion recognition in conversation task that uses context and multiple modalities to predict emotion label for a given utterance. |
| Outcome: | The proposed method outperforms the state-of-the-art methods on three multimodal datasets. |
Amanda: Adaptively Modality-Balanced Domain Adaptation for Multimodal Emotion Recognition (2024.findings-acl)
Copied to clipboard
| Challenge: | Emotion recognition is a multimodal learning method that can be used for data scarcity. |
| Approach: | They propose to use Adaptively modality-balanced domain adaptation to balance the alignment of different modalities for multimodal emotion recognition. |
| Outcome: | The proposed model outperforms competing models on common datasets on multimodal emotion recognition. |
Joyful: Joint Modality Fusion and Graph Contrastive Learning for Multimoda Emotion Recognition (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing graph-based methods fail to depict global contextual features and local diverse unimodal features in a dialogue. |
| Approach: | They propose a method for joint modality fusion and graph contrastive learning for multimodal emotion recognition using a multimodal fusion mechanism and a graph contrastative learning framework. |
| Outcome: | The proposed method improves multimodal emotion recognition on unbalanced and small-scale emotional datasets. |